Point-to-point distance measurements in 3D camera images

ABSTRACT

Systems and methods for determining point-to-point distances from 3D image data. In some embodiments, two measure points, for example specified by a user, represent endpoints on an object of interest within an image frame. Assuming all points lying between these endpoints also belong to the object of interest, additional 3D data associated with points that lie along a measurement line defined by the measure points may be leveraged to provide a robust distance measurement. In some embodiments, total least squares fitting is performed, for example through Robust Principal Component Analysis (RPCA) to identify linear structures within the set of the 3D coordinates on the measurement line. In some exemplary embodiments, the minimum covariance determinant (MCD) estimator of the covariance matrix of the data is computed for a highly robust estimate of multivariate location and multivariate scatter.

BACKGROUND

A digital camera is a component often included in commercial electronicmedia device platforms. Digital cameras are now available in wearableform factors (e.g., video capture earpieces, video capture headsets,video capture eyeglasses, etc.), as well as embedded within smartphones,tablet computers, and notebook computers, etc. Three-dimensional (3D)cameras are becoming more common, and can now be found on many mobiledevices/platforms, including vehicles. These devices provide enhancedentertainment and utility experiences to an end user. For example,photography and vehicle control systems may be enhanced by depth (i.e.,range) information output from the 3D camera. Existing technology togenerate 3D camera images include stereo or multi-cameras, structuredlight systems comprising projected light patterns and cameras,time-of-flight (TOF) cameras, etc.

An important application enabled by 3D cameras is point-to-pointmeasurement where the distance between two points on an image that aretypically specified by the user, is measured in real world units, suchas centimeters, etc. The simplest way to measure such a point-to-pointdistance is to utilize the range information, along with geometriccalibration information, to compute 3D coordinates of the measurementpoints. Measurement of the intervening distance between the selectedmeasurement points can then be computed. Such an approach to measurementsuffers from several drawbacks, however. For example, range values areoften noisy, introducing significant error when a pair of points arespecified. Also, the technique is also susceptible to imprecise userinput of the measurement location.

A more robust point-to-point distance measurement in 3D camera images istherefore advantageous.

BRIEF DESCRIPTION OF THE DRAWINGS

The material described herein is illustrated by way of example and notby way of limitation in the accompanying figures. For simplicity andclarity of illustration, elements illustrated in the figures are notnecessarily drawn to scale. For example, the dimensions of some elementsmay be exaggerated relative to other elements for clarity. Further,where considered appropriate, reference labels have been repeated amongthe figures to indicate corresponding or analogous elements. In thefigures:

FIG. 1 is a schematic of point-to-point distance measurements in 3Dcamera images, in accordance with some embodiments;

FIG. 2A is a flow diagram of a method for point-to-point distancemeasurement, in accordance with some embodiments;

FIG. 2B is a schematic of a point-to-point gap distance measurement in3D camera images, in accordance with some embodiments;

FIG. 3 is a flow diagram of a method for point-to-point distancemeasurement, in accordance with some embodiments;

FIG. 4 illustrates an image capture device including a point-to-pointdistance measurement pipeline, in accordance with some embodiments;

FIG. 5 is a block diagram of an embodiment of a processor having one ormore processor cores, an integrated memory controller, and an integratedgraphics processor, according to some embodiments;

FIG. 6 is a diagram of an exemplary ultra-low power system employing apoint-to-point measurement device, in accordance with some embodiments;and

FIG. 7 is a diagram of an exemplary mobile handset platform, arranged inaccordance with some embodiments.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

One or more embodiments are described with reference to the enclosedfigures. While specific configurations and arrangements are depicted anddiscussed in detail, it should be understood that this is done forillustrative purposes only. Persons skilled in the relevant art willrecognize that other configurations and arrangements are possiblewithout departing from the spirit and scope of the description. It willbe apparent to those skilled in the relevant art that techniques and/orarrangements described herein may be employed in a variety of othersystems and applications beyond what is described in detail herein.

Reference is made in the following detailed description to theaccompanying drawings, which form a part hereof and illustrate exemplaryembodiments. Further, it is to be understood that other embodiments maybe utilized and structural and/or logical changes may be made withoutdeparting from the scope of claimed subject matter. Therefore, thefollowing detailed description is not to be taken in a limiting senseand the scope of claimed subject matter is defined solely by theappended claims and their equivalents.

In the following description, numerous details are set forth, however,it will be apparent to one skilled in the art, that embodiments may bepracticed without these specific details. Well-known methods and devicesare shown in block diagram form, rather than in detail, to avoidobscuring more significant aspects. References throughout thisspecification to “an embodiment” or “one embodiment” mean that aparticular feature, structure, function, or characteristic described inconnection with the embodiment is included in at least one embodiment.Thus, the appearances of the phrase “in an embodiment” or “in oneembodiment” in various places throughout this specification are notnecessarily referring to the same embodiment. Furthermore, theparticular features, structures, functions, or characteristics describedin the context of an embodiment may be combined in any suitable mannerin one or more embodiments. For example, a first embodiment may becombined with a second embodiment anywhere the particular features,structures, functions, or characteristics associated with the twoembodiments are not mutually exclusive.

As used in the description of the exemplary embodiments and in theappended claims, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items.

As used throughout the description, and in the claims, a list of itemsjoined by the term “at least one of” or “one or more of” can mean anycombination of the listed terms. For example, the phrase “at least oneof A, B or C” can mean A; B; C; A and B; A and C; B and C; or A, B andC.

The terms “coupled” and “connected,” along with their derivatives, maybe used herein to describe functional or structural relationshipsbetween components. It should be understood that these terms are notintended as synonyms for each other. Rather, in particular embodiments,“connected” may be used to indicate that two or more elements are indirect physical, optical, or electrical contact with each other.“Coupled” may be used to indicated that two or more elements are ineither direct or indirect (with other intervening elements between them)physical, optical, or electrical contact with each other, and/or thatthe two or more elements co-operate or interact with each other (e.g.,as in a cause an effect relationship).

Some portions of the detailed descriptions provide herein are presentedin terms of algorithms and symbolic representations of operations ondata bits within a computer memory. Unless specifically statedotherwise, as apparent from the following discussion, it is appreciatedthat throughout the description, discussions utilizing terms such as“calculating,” “computing,” “determining” “estimating” “storing”“collecting” “displaying,” “receiving,” “consolidating,” “generating,”“updating,” or the like, refer to the action and processes of a computersystem, or similar electronic computing device, that manipulates andtransforms data represented as physical (electronic) quantities withinthe computer system's circuitry including registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

While the following description sets forth embodiments that may bemanifested in architectures, such system-on-a-chip (SoC) architecturesfor example, implementation of the techniques and/or arrangementsdescribed herein are not restricted to particular architectures and/orcomputing systems, and may be implemented by any architecture and/orcomputing system for similar purposes. Various architectures employing,for example, multiple integrated circuit (IC) chips and/or packages,and/or various computing devices and/or consumer electronic (CE) devicessuch as set-top boxes, smartphones, etc., may implement the techniquesand/or arrangements described herein. Further, while the followingdescription may set forth numerous specific details such as logicimplementations, types and interrelationships of system components,logic partitioning/integration choices, etc., claimed subject matter maybe practiced without such specific details. Furthermore, some materialsuch as, for example, control structures and full software instructionsequences, may not be shown in detail in order not to obscure thematerial disclosed herein.

Certain portions of the material disclosed herein may be implemented inhardware, for example as logic circuitry in an image processor. Certainother portions may be implemented in hardware, firmware, software, orany combination thereof. At least some of the material disclosed hereinmay also be implemented as instructions stored on a machine-readablemedium, which may be read and executed by one or more processors(graphics processors and/or central processors). A machine-readablemedium may include any medium and/or mechanism for storing ortransmitting information in a form readable by a machine (e.g., acomputing device). For example, a machine-readable medium may includeread only memory (ROM); random access memory (RAM); magnetic diskstorage media; optical storage media; flash memory devices; electrical,optical, acoustical, or other similarly non-transitory, tangible media.

One or more system, apparatus, method, and computer readable media isdescribed below for determining point-to-point distances from 3D imagedata. In some embodiments, two measure points, for example specified bya user, represent endpoints on an object of interest within the imageframe. Assuming all points lying between these endpoints also belong tothe object of interest, additional 3D data associated with points thatlie along a measurement line defined by the measure points may beleveraged according to embodiments described herein to provide a robustdistance measurement.

FIG. 1 is a schematic of point-to-point distance measurements in 3Dcamera images, in accordance with some embodiments. In the illustratedembodiment, a camera platform includes an array camera 115. In someembodiments, array camera 115 is a component of a mobile computingdevice having a plurality of camera hardware modules (CM) 110, 111, and112 with a predetermined baseline relationship, Although in theexemplary embodiments three camera hardware modules are illustrated, anynumber of camera hardware modules and/or image sensors may be includedin an array camera.

Each of the plurality of camera modules 110, 111, 112 may collect aninput image captured from a different camera viewpoint. In exemplaryembodiments, the images captured of the world scene 101 from differentviewpoints are captured at substantially the same instant of time suchthat they contain image data for a given scene. One of the three imageframes may be designated as a reference, and combined with informationfrom the other frames into an image frame 120 having depth or range (z)information, or disparity information indicative of depth, for each 2D(x,y) spatial position within the image frame. For example, where CM 110has a higher resolution (e.g., 8 megapixel, or more) than camera modules111, 112 (e.g., 720p, HD, etc.), CM 110 may provide a default referenceRGB image. CM 111 and CM 112 may be considered supplemental to thereference module and are each associated with predetermined baselinevector (length and direction) from camera module 110 to generate depthdata (D). The input image may then be associated with RGB-D data foreach 2D spatial position with the input image frame. In an exemplaryembodiment where camera modules 110, 111, and 112 are on a mobileplatform, the baseline vector between the reference camera module andeach supplemental camera module may have a length of tens of millimetersto tens of centimeters, depending on the form factor. In otherembodiments, where camera modules 110, 111, 112 are separateinfrastructure fixtures, baseline lengths may be on the order of meters.In one exemplary mobile device embodiment, camera modules 111, 112 arespaced by a known distance on opposite sides of reference camera module110.

As further illustrated in FIG. 1, input image frame 120 includes objects140 and 150. Measure points 130A and 130B are received, for examplethrough user input. The user's goal in such a situation may be todetermine the real world physical width of object 140 from input imageframe 120. As such, the measure point inputs 130A and 130B are employedas endpoints of a measurement line 130 extending across object 140. As afurther illustration, measure points 135A and 135B are also received,for example through a second user input. The user's goal in this secondsituation may be to determine the real world physical width of a gap orspace between measure points 135A and 135B. As such, the second measurepoint inputs 135A and 135B are again employed as endpoints of ameasurement line 135 extending between object 140 and object 150. In thefirst example, it is helpful to employ input image data along the entirelength of measurement line 130 under the assumption that measure points130A, 130B lie along edges of an object such that the entire measurementline 130 is spanning the object 140 to be measured. Such an assumptionis based on the logic that a user often desires measurement data for asingle object at a time. In some embodiments described herein, arefinement of the 3D spatial coordinates corresponding to measure points130A and 130B is performed based on additional information obtained byanalyzing the 3D structure of a plurality of points along themeasurement line.

In the second example however, it may not be helpful to employ inputimage data from the entire length of measurement line 135 because it isa gap measurement and many different background objects may be presentwithin the gap, each associated with a different depth value. Therefore,in some further embodiments herein, measure points 135A, 135B are notrefined based on additional information obtained by analyzing the 3Dstructure of a plurality of points along the measurement line.

FIG. 2A is a flow diagram of a method 201 for determining point-to-pointdistances from 3D image data. In exemplary embodiments, method 201 isimplemented by one or more computer processors, for example integratedinto a 3D camera platform as described further below. Method 201 beginswith receiving input image data associated with an image frame atoperation 205. The input image data is 3D image data including depthinformation, in addition to color information (e.g., intensity for eachof a plurality of color channels in any color space, such as RGB,YP_(B)P_(R), YC_(B)C_(R), or the like). There may be a depth ordisparity value associated with each pixel of an image frame, or somedepth/disparity values may be absent in the event of a camera occlusionand/or textureless surface, etc. Even where a depth value is associatedwith each pixel of an input image frame, the depth data is expected tobe noisy as a result of upstream image processing employed to determinethe depth values.

In some embodiments, depth information received at operation 205 is inthe form of a depth map correlated with a plurality of pixels p_(i),each having an image coordinate x_(i), y_(i) associated with the inputimage frame. In other embodiments, the depth information received atoperation 205 is in the form of a disparity map correlated with aplurality of pixels, each having an image coordinate x_(i), y_(i)associated with the input image. A disparity value associated with apixel indicates the correspondence of the pixel in one image (e.g.,collected by camera module 110 in FIG. 1) to a pixel in another image(e.g., collected by camera module 111). The disparity estimation may beby any technique, as embodiments herein are not limited in this respect.

For some embodiments where a 3D (X,Y,Z) spatial coordinate map is notreceived as an input at operation 205, a “world 3D coordinate” map ofthe image data is generated at operation 205. Any mapping functions maybe utilized to determine 3D spatial coordinates from a disparity valueat the corresponding pixel position, for example based on predeterminedcamera parameters received at operation 205. For each image pointp=(x,y), there is associated range information Z=R(x,y) that is aprojection of the 3D point P given by:

$\begin{matrix}{{P = {\begin{bmatrix}X \\Y \\Z\end{bmatrix} = {K^{- 1}\begin{bmatrix}x \\y \\z\end{bmatrix}}}},} & (1)\end{matrix}$where the projection matrix K is associated with intrinsic cameracalibration information. For many consumer digital cameras withspherical lensing, f_(x)=f_(y) and skew s=0, and the projection matrixsimplifies to:

$\begin{matrix}{{K = \begin{bmatrix}f_{x} & s & c_{x} \\0 & f_{y} & c_{y} \\0 & 0 & 1\end{bmatrix}},} & (2)\end{matrix}$where c_(x), c_(y) denotes the principal point.

At operation 210, measure points associated with the input image frameare received. In exemplary embodiments, measure points are received aspairs: p₁ and p₂. As one example, a user selection of two measure pointsp₁, p₂ may be received from a touch input sampled by a touch displaypresenting the input image frame to the user. With p₁=(x₁,y₁) andp₂=(x₂,y₂), each measure point p₁ and p₂ is associated with a 3D spatialcoordinate P₁, and P₂, respectively, as determined from Eq. (1) and Eq.(2).

At operation 230, a set of 3D spatial coordinates is similarlydetermined for pixel positions along a measurement line defined by themeasure points p₁, p₂. Image pixel positions along the measurement linemay be sampled at the closest integer coordinates in the image frame. Insome embodiments, to maximize the number of sampling points, the 3Dimage data is sampled along the x-dimension if|p₁(x)−p₂(x)|>|p₁(y)−p₂(y)|, and along they-dimension otherwise.Depending on the method employed to compute depth, 3D coordinates can beobtained for all or a subset of points along the measurement line. Insome embodiments, the set points {L_(i)} on the measurement line arestored to an electronic memory, where L_(i)=X_(i), Y_(i), Z_(i), andi=1, 2, . . . N. Any missing depth values are not included in set ofmeasurement line points {L_(i)}.

At operation 240, the set of measurement line points {L_(i)} arefiltered to account for outliers. In exemplary embodiments, the filteroperation 240 is based on the logic that many real world objects a usermight measure have a profile that is linear along the measurement axis.Thus many objects, if not too close to the camera, will have a planarcharacter. For example, 3D points on a human often appear planar in manydepth measurements. This is also true with many commonly occurringpolygonal objects in the world such as buildings, furniture, cars,photographs, etc. In some embodiments, the filtering operation 240employs this assumption to enhance the accuracy of the measurementbetween measure points p₁, p₂ by finding linear structure(s) in the setof measurement line points {L_(i)}. A subset of the set of measurementline points {L_(i)} having inadequate linear structure is removed,ignored, or otherwise discounted as containing outliers.

At operation 250, the 3D coordinates for the measure points (P₁, P₂) arethen mapped to 3D coordinates associated with predicted, or “refined”measurement points (P₁′, P₂′), respectively. The refinement is based ona function derived from the measurement line points having sufficientlinear structure. This measurement line function may be employed todetermine the predicted measure point coordinates P₁′, P₂′. In someembodiments, the predicted measure point coordinates P₁′, P₂′ areemployed to determine whether the measurement line is associated with apoint-to-point gap measurement rather than a point-to-point objectmeasurement. Where input measure points p₁, p₂ span a gap betweendifferent objects, such as the distance between of a doorway asillustrated in FIG. 1, refinement by filtering on the basis of linearityover the measurement line may not hold significant advantages. Method201 therefore further illustrates some embodiments that employ gapmeasurement detection. Use of the predicted measure point coordinatesP₁′, P₂′ may then be conditioned upon an outcome of the gap detectionalgorithm. In response to detecting a gap, the point-to-point distancemeasure may be determined independent of predicted measure pointcoordinates P₁′, P₂′. In the absence of detecting a gap, thepoint-to-point distance measure may be determined based on predictedmeasure point coordinates P₁′, P₂′.

In the exemplary embodiment, at operation 260 the measurement line P₁P₂is extended by end segments of a predetermined, fixed length. FIG. 2B isa schematic of a point-to-point gap distance measurement, illustrating afirst end segment 235A extending away from P₁ (135A) along measurementline P₁P₂ (135), and a second end segment 235B extending away from P₂(135B) along measurement line P₁P₂ (135). At operation 270, a depthassociated with each of the end segments is compared to the nearestpredicted measure point depth Z′. In advantageous embodiments, a robustestimate of a first end segment depth Z₁ ^(ext) is determined from thedepth values in the first end segment. As one example, a biweight orbisquare function may be employed to estimate depth Z₁ ^(ext). The sametechnique may be applied to arrive at an estimate of the second endsegment depth Z₂ ^(ext).

Method 201 then proceeds to either operation 280 or 290 in response tothe outcome of the comparison between Z₁′ and Z₁ ^(ext) and thecomparison between Z₂′ and Z₂ ^(ext). In response to at least one of theend segments having an equal or greater depth than that of the nearestpredicted measure point depth Z′, method 201 proceeds to operation 280where a point-to-point object distance measurement is determined basedon the predicted measure point coordinates P₁′, P₂′. Although any knowntechnique may be employed at operation 280, in some exemplaryembodiments, the Euclidean distance between predicted measure pointcoordinates P₁′, P₂′ is computed as:d=∥P′ ₁ −P′ ₂∥₂=√{square root over ((X ₁ −X ₂)²+(Y ₁ −Y ₂)²+(Z ₁ −Z₂)²)}.  (3)

The measured distance d computed based on the predicted measure pointsis of improved accuracy, removing error from user input of measurepoints p₁, p₂ that might be obtained through a pointer or touchinterface. Often, in the case of measure points p₁, p₂ being specifiedfor measurement of an object of interest, points on foreground orbackground regions adjoining the object may be inadvertently included.3D coordinate data filtered for linearity will remove such errors.

In response to both the end segments having less depth than that of thenearest predicted measure point depth Z′ (i.e., Z₁ ^(ext)<Z₁′ AND Z₂^(ext)<Z₂′), a gap measurement is detected. Method 201 then proceeds tooperation 290 where a point-to-point gap distance measurement isdetermined based on the measure point coordinates P₁, P₂. Although anyknown technique may be employed at operation 290, in some exemplaryembodiments, the Euclidean distance between measure point coordinatesP₁, P₂ is computed as described above for the predicted measure pointcoordinates P₁′, P₂′.

Method 201 is then complete upon storing and/or outputting the distanced as a point-to-point measurement at operation 299. Distance d may bestored as metadata associated with the input measure points p₁, p₂,and/or as metadata associated with the input image. In some embodiments,the distance d is stored to an electronic memory in association with themeasure points p₁, p₂ received at operation 210. In some otherembodiments, the distance d is stored to an electronic memory inassociation with the input image received at operation 205. In stillother embodiments, the distance d is stored to an electronic memory inassociation with the measure points p₁, p₂, and in association with theinput image received at operation 205.

In some embodiments, a robust total least square (TLS) fitting of the 3Dcoordinates in the measurement line set {L_(i)} is performed to accountfor outliers, for example at operation 240 above, and to arrive at thepredicted measure point coordinates P₁′, P₂′. Such a technique, or theequivalent, is appropriate because the variables along all 3 directions(X,Y,Z) may suffer from significant error. FIG. 3 is a flow diagram of amethod 301 for point-to-point distance measurement, in accordance withsome exemplary embodiments. Method 301 further illustrates an exemplaryembodiment of method 201.

Method 301 begins with collecting input image data with a 3D cameramodule at operation 305. In some embodiments, the 3D camera module isthe array camera 115 (FIG. 1) introduced above. In other embodiments,the input image data is collected with a time of flight (TOF) camera. Instill other embodiments, a structured light source (illuminator) isemployed at operation 305 to determine a depth data map over the inputimage frame. Other known 3D cameras and depth mapping techniques mayalso be employed at operation 305 to collect 3D data for an input imageframe.

At operation 310, the input image frame collected at operation 305 ispresented on a display screen. In some exemplary embodiments, the inputimage frame is presented on a display screen integrated into a sameplatform as the 3D camera, such as, but not limited to a mobile deviceplatform. At operation 315, measure points p₁, p₂ are received through agraphical user interface (GUI) implemented by the 3D camera platform.For example, in some embodiments, user touches are received at operation315 while the display screen is presenting of the input image atoperation 310.

At operation 320, 3D spatial coordinate data is sampled along ameasurement line extending between the two measure points. From thissampling, the set of measurement line points {L_(i)} is generated, asdescribed above.

In some embodiments, TLS fitting is performed through PrincipalComponent Analysis (PCA) to identify linear structures within {L_(i)}.However, traditional PCA is sensitive to outliers in the data andtherefore is not well-suited to a measurement problem where outliers inthe depth data are commonly observed. Robust PCA (RPCA) is thereforeadvantageous. At operation 325 therefore, a robust estimate of thecovariance matrix is computed to address the difficulty posed byoutliers. In some exemplary embodiments, the minimum covariancedeterminant (MCD) estimator of the covariance matrix of the data iscomputed. MCD provides a highly robust estimate of multivariate locationT and multivariate scatter S. The objective of the MCD algorithm is tofind h observations out of N included in the measurement line set{L_(i)} whose covariance matrix has the lowest determinant. While anyknown MCD algorithm may be employed at operation 325, in someembodiments the Fast MCD algorithm is employed to generate the MCDsolution.

The parameter h, which determines the breakdown point of the Fast MCDestimator may be predetermined for a given camera platform. Differentdepth measurement systems have different accuracies and these may beconsidered in selecting h, which directly corresponds to the expectednumber of outliers for the system. In some exemplary embodiments where adisparity estimation is determined with a three camera system, h mightbe 0.5N, or less. For larger array camera systems, h may be increasedwith expected improvements in disparity accuracy. This is also true ofstructured light or time-of-flight systems that use active illuminationto more accurately measure depth.

The Fast MCD algorithm advantageously allows for exact fit situationswhere h or more observations lie on a hyperplane, while also yieldingrobust estimates of location and scatter (which is singular when h ormore observations lie on a hyperplane). In response to non-singularscatter, method 301 proceeds to operation 330 where an Eigendecomposition of the multivariate scatter S is performed as S=VΛ^(T) tocompute the principal components of the data. The first principalcomponent, which is the Eigen vector corresponding to the largest Eigenvalue, may be denoted v₀. At operation 345, P₁ and P₂ are projected ontothe principal component v₀ to obtain the projected coordinates P₁′ andP₂′, as:P′ ₁ =T+[(P ₁ −T)·v ₀ ]v ₀,  (4)P′ ₂ =T+[(P ₂ −T)·v ₀ ]v ₀.  (5)

If instead, all points fall on the hyperplane, method 335 proceeds tooperation 335 where an equation of the hyperplane is computed. Inexemplary embodiments the Fast MCD algorithm is further employed atoperation 335 compute the hyperplane equation aX+bY+cZ+d=0. At operation340 the hyperplane equation is evaluated at measure point coordinatesP₁, P₂. If a; b=0, the object being measured has a single depthmeasurement and that the hyperplane simplifies to

$Z = {- {\frac{d}{c}.}}$The X coordinate of P₁ and P₂ is then set to

$- \frac{d}{c}$and the refined points P₁′, P₂′ are re-computed using this predicteddepth measurement in Equation (1). In response to either a or b notequaling 0, the measure point coordinates P₁ and P₂ are projected onto aline to compute the predicted measure point coordinates P₁′ and P₂′, asgiven below for P₁′:

$\begin{matrix}{{t_{1} = \frac{{- {aX}_{1}} - {bY}_{1} - {cZ}_{1} - d}{a^{2} + b^{2} + c^{2}}},} & (6) \\{P_{1}^{\prime} = \begin{bmatrix}{{at}_{1} + X_{1}} \\{{bt}_{1} + Y_{1}} \\{{ct}_{1} + Z_{1}}\end{bmatrix}} & (7)\end{matrix}$

With the predicted measure point coordinates P₁′ and P₂′ determined,method 301 continues at operation 365 wherein the distance between thepredicted measure points is determined. In some embodiments, for exampleby applying Equation (3). Method 301 is completed with storing thedistance d to an electronic memory as the point-to-point distance forthe input measure points received at operation 315. The distance d maybe stored in association with either or both the input image and inputmeasure points. The distance d may further be output to a display screenof the camera device or otherwise presented or transmitted to a user.

FIG. 4 further illustrates how a point-to-point measurement device maybe integrated with various other components of an image and/or videocapture device 400 to provide enhanced camera output suitable forpoint-to-point measurement determinations. Image capture device 400 forexample may be a portion of a mobile computing device platform. A mobilecomputing device may refer to any device having a processing system anda mobile power source or supply, such as one or more batteries, forexample. Examples of a mobile computing device may include a laptopcomputer, tablet, touch pad, portable computer, handheld computer,palmtop computer, personal digital assistant (PDA), cellular telephone,combination cellular telephone/PDA, television, smart device (e.g.,smartphone, tablet or smart television), mobile internet device (MID),messaging device, data communication device, and so forth. Examples of amobile computing device also include computers and/or mediacapture/transmission devices configured to be worn by a person, such asa wrist computer, finger computer, ring computer, eyeglass computer,belt-clip computer, arm-band computer, shoe computers, clothingcomputers, and other wearable computers. In various embodiments, forexample, a mobile computing device may be implemented as a smart phonecapable of executing computer applications, as well as voicecommunications and/or data communications. Although some embodiments maybe described with a mobile computing device implemented as a smart phoneby way of example, it may be appreciated that other embodiments may beimplemented using other wireless mobile computing devices as well. Theembodiments are not limited in this context.

Image capture device 400 includes hardware camera modules 110, 111, and112. In the exemplary embodiment, CM 110 further includes a RGB(NIR)camera sensor 458 while CM 111 and 112 each include a RGB(NIR) camerasensor 459. Sensor 458 may be a HD, FHD, QXGA, WQXGA, QSXGA, or UHDformat digital image device, for example. In some embodiments, sensor458 has at least 8-megapixel resolution. Sensors 459 may be a HD, FHD,QXGA, WQXGA, QSXGA, or UHD format digital image device, for example. Insome embodiments, sensors 459 have a lower pixel resolution than sensor458, for example 1-5 mega pixel. Image/video capture device 400 maytherefore generate three image frames concurrently, for example toprovide both RGB image data and image depth data for an input image.

Camera sensors 458, 459 may provide a color resolution of 8 bits, ormore per pixel, and be operable to capture continuous video framesprogressively. Sensor 458 may have a pixel frequency of 170 MHz, ormore. Sensors 458, 459 may include an RGB Bayer color filter, an analogamplifier, an A/D converter, other components to convert incident lightinto a digital signal corresponding to raw image data. Sensors 458, 459may be controlled to operate a rolling shutter or electronic focal planeshutter process where pixels are read out progressively in aline-sequential fashion for a frame. CM 110, 111, and 112 may eachoutput raw data associated with consecutively exposed frames inconformance with any known streaming protocol, such as a MIPI.

In the exemplary embodiment, raw image/video data output by CM 111 and112 is input to ISP 475. ISP 475 is to receive and analyze frames of rawvideo data during the horizontal and/or vertical blanking periodsassociated with CM 111 and 112. During raw image data processing of RGBimage data, ISP 475 may perform one or more of color space conversion,noise reduction, pixel linearization, and shading compensation, forexample. In some embodiments, raw image data is passed through ISP 475to be processed downstream by a programmable microprocessor 450.

Image data output by ISP 475 may be buffered and queued as input imagedata ready for further image processing, such as point-to-point 3D imagemeasurement, for example in accordance with one or more of theembodiments described elsewhere herein. In embodiments, processor(s) 450includes logic to perform point-to-point measurement operations andexecute algorithms described elsewhere herein. In embodiments,processor(s) 450 includes logic to perform the point-to-pointmeasurement operations and execute algorithms described elsewhereherein. In some embodiments, processor(s) 450 includes logic to performone or more of the operations of method 201 (FIG. 2) and/or method 301(FIG. 3). In some embodiments, processor(s) 450 includes pixel samplinglogic 401 to determine 3D spatial coordinates for points along ameasurement line define by two endpoint inputs. In some embodiments,pixel sampling logic 401 is implemented with programmable circuitryincluding registers that have been configured through softwareinstruction(s). In some embodiments, processor(s) 450 includes 3Dcoordinate linear fitting logic 402 to implement a robust technique foridentifying linear structure within the sampled set of 3D coordinates,for example with and RPCA and/or MCD algorithms, as described elsewhereherein. In some embodiments, 3D coordinate linear fitting logic 402 isimplemented with programmable circuitry including registers that havebeen configured through software instruction(s). In some embodiments,processor(s) 450 includes measurement logic 403 to determine apoint-to-point distance between predicted measure point coordinates, forexample as described elsewhere herein. In some embodiments, measurementlogic 403 is implemented with programmable circuitry including registersthat have been configured through software instruction(s).

Both software and hardware implementations may be well suited toimplementing point-to-point measurements in accordance with embodimentsdescribed herein. For hardware implementations, pixel sampling logic401, and/or 3D coordinate linear fitting logic 402, and/or measurementlogic 403 may be implemented by fixed function logic, for exampleprovided by ISP 475. For software implementations, any knownprogrammable processor, including a core of processor(s) 450, anexecution unit of a graphics processor, or vector processor, may beutilized to implement the pixel sampling logic 401 and/or 3D coordinatelinear fitting logic 402, and/or measurement logic 403. Processor(s) 450may be solely responsible for generating point-to-point measurement datafrom input image data received from ISP 475. In one exemplaryembodiment, pixel sampling logic 401 and/or 3D coordinate linear fittinglogic 402, and/or measurement logic 403 are invoked through the userspace of a software stack instantiated by processor(s) 450. In someembodiments, processor(s) 450 executes pixel sampling logic 401, and/or3D coordinate linear fitting logic 402, and/or measurement logic 403instantiated in a kernel space of the software stack. In someembodiments, processor(s) 450 is programmed with instructions stored ona computer readable media to cause the processor to perform one or moreof pixel sampling along a measurement line, pixel filtering based onlinear structure, or distance determination based on predicted measurepoint coordinates, for example as described elsewhere herein.

As further illustrated in FIG. 4, point-to-point measurement data may beoutput to storage/display/transmission pipeline 495. In one exemplarystorage pipeline embodiment, point-to-point measurement data is writtento electronic memory 420 (e.g., DDR, etc.) to supplement stored inputimage data. Memory 420 may be separate or a part of a main memory 410accessible to processor 450. Alternatively, or in addition,storage/display/transmission pipeline 495 is to transmit point-to-pointmeasurement data and/or input image data off video/image capture device400.

FIG. 5 block diagrams a data processing system 500 that may be utilizedto processing an input image to determine a point-to-point measurementdistance. Data processing system 500 includes one or more processors 450and one or more graphics processors 501, and may be implemented in asingle processor desktop system, a multiprocessor workstation system, ora server system having a large number of processors 450 or processorcores 507. In another embodiment, the data processing system 500 is asystem-on-a-chip (SoC) integrated circuit for use in mobile, handheld,or embedded devices.

An embodiment of data processing system 500 can include, or beincorporated within a server-based gaming platform, a game console,including a game and media console, a mobile gaming console, a handheldgame console, or an online game console. In some embodiments, dataprocessing system 500 is a mobile phone, smart phone, tablet computingdevice or mobile Internet device. Data processing system 500 can alsoinclude, couple with, or be integrated within a wearable device, such asa smart watch wearable device, smart eyewear device, augmented realitydevice, or virtual reality device. In some embodiments, data processingsystem 500 is a television or set top box device having one or moreprocessors 502 and a graphical interface generated by one or moregraphics processors 508.

In some embodiments, the one or more processors 502 each include one ormore processor cores 507 to process instructions which, when executed,perform operations for system and user software. In some embodiments,each of the one or more processor cores 507 is configured to process aspecific instruction set 509. In some embodiments, instruction set 509may facilitate Complex Instruction Set Computing (CISC), ReducedInstruction Set Computing (RISC), or computing via a Very LongInstruction Word (VLIW). Multiple processor cores 507 may each process adifferent instruction set 509, which may include instructions tofacilitate the emulation of other instruction sets. Processor core 507may also include other processing devices, such a Digital SignalProcessor (DSP).

In some embodiments, the processor 502 includes cache memory 504.Depending on the architecture, the processor 502 can have a singleinternal cache or multiple levels of internal cache. In someembodiments, the cache memory is shared among various components of theprocessor 502. In some embodiments, the processor 502 also uses anexternal cache (e.g., a Level-3 (L3) cache or Last Level Cache (LLC))(not shown), which may be shared among processor cores 507 using knowncache coherency techniques. A register file 506 is additionally includedin processor 502 which may include different types of registers forstoring different types of data (e.g., integer registers, floating pointregisters, status registers, and an instruction pointer register). Someregisters may be general-purpose registers, while other registers may bespecific to the design of the processor 502.

In some embodiments, processor 502 is coupled to a processor bus 510 totransmit data signals between processor 502 and other components insystem 500. System 500 has a ‘hub’ system architecture, including amemory controller hub 516 and an input output (I/O) controller hub 530.Memory controller hub 516 facilitates communication between a memorydevice and other components of system 500, while I/O Controller Hub(ICH) 530 provides connections to I/O devices via a local I/O bus.

Memory device 520 can be a dynamic random access memory (DRAM) device, astatic random access memory (SRAM) device, flash memory device, or someother memory device having suitable performance to serve as processmemory. Memory 520 can store data 522 and instructions 521 for use whenprocessor 502 executes a process. Memory controller hub 516 also coupleswith an optional external graphics processor 512, which may communicatewith the one or more graphics processors 508 in processors 502 toperform graphics and media operations.

In some embodiments, ICH 530 enables peripherals to connect to memory520 and processor 502 via a high-speed I/O bus. The I/O peripheralsinclude an audio controller 546, a firmware interface 528, a wirelesstransceiver 526 (e.g., Wi-Fi, Bluetooth), a data storage device 524(e.g., hard disk drive, flash memory, etc.), and a legacy I/O controllerfor coupling legacy (e.g., Personal System 2 (PS/2)) devices to thesystem. One or more Universal Serial Bus (USB) controllers 542 connectinput devices, such as keyboard and mouse 544 combinations. A networkcontroller 534 may also couple to ICH 530. In some embodiments, ahigh-performance network controller (not shown) couples to processor bus510.

FIG. 6 is a diagram of an exemplary ultra-low power system 600 employinga point-to-point measurement device, in accordance with one or moreembodiment. System 600 may be a mobile device although system 600 is notlimited to this context. System 600 may be incorporated into a wearablecomputing device, laptop computer, tablet, touch pad, handheld computer,palmtop computer, cellular telephone, smart device (e.g., smart phone,smart tablet or mobile television), mobile internet device (MID),messaging device, data communication device, and so forth. System 600may also be an infrastructure device. For example, system 700 may beincorporated into a large format television, set-top box, desktopcomputer, or other home or commercial network device.

System 600 includes a device platform 602 that may implement all or asubset of the various point-to-point measurement methods and any of thelogic blocks/circuitry described above in the context of FIG. 2-FIG. 5.In various exemplary embodiments, video processor 615 executes at leastone of pixel sampling, robust linear fitting, and distance measurementlogic, for example as described above. Video processor 615 includeslogic circuitry 607 implementing point-to-point measurement based on 3Ddata from an input image, for example as described elsewhere herein. Insome embodiments, one or more computer readable media may storeinstructions, which when executed by CPU 610 and/or video processor 615,cause the processor(s) to execute one or more point-to-point measurementmethod, such as any of those described in detail above. A point-to-pointmeasurement may then be stored in memory 612.

In embodiments, device platform 602 is coupled to a human interfacedevice (HID) 620. Platform 602 may collect raw image data with CM 110,111, 112, which is processed and output to HID 620. A navigationcontroller 650 including one or more navigation features may be used tointeract with, for example, device platform 602 and/or HID 620. Inembodiments, HID 620 may include any television type monitor or displaycoupled to platform 602 via radio 618 and/or network 660. HID 620 mayinclude, for example, a computer display screen, touch screen display,video monitor, television-like device, and/or a television to receivetouch inputs while an input image is displayed on display 622.

In embodiments, device platform 602 may include any combination of CM110-112, chipset 605, processors 610, 615, memory/storage 612,applications 616, and/or radio 618. Chipset 605 may provideintercommunication among processors 610, 615, memory 612, videoprocessor 615, applications 616, or radio 618.

One or more of processors 610, 615 may be implemented as one or moreComplex Instruction Set Computer (CISC) or Reduced Instruction SetComputer (RISC) processors; x86 instruction set compatible processors,multi-core, or any other microprocessor or central processing unit(CPU).

Memory 612 may be implemented as a volatile memory device such as, butnot limited to, a Random Access Memory (RAM), Dynamic Random AccessMemory (DRAM), or Static RAM (SRAM). Memory 612 may also be implementedas a non-volatile storage device such as, but not limited to flashmemory, battery backed-up SDRAM (synchronous DRAM), magnetic memory,phase change memory, and the like.

Radio 618 may include one or more radios capable of transmitting andreceiving signals using various suitable wireless communicationstechniques. Such techniques may involve communications across one ormore wireless networks. Example wireless networks include (but are notlimited to) wireless local area networks (WLANs), wireless personal areanetworks (WPANs), wireless metropolitan area network (WMANs), cellularnetworks, and satellite networks. In communicating across such networks,radio 618 may operate in accordance with one or more applicablestandards in any version.

In embodiments, system 600 may be implemented as a wireless system, awired system, or a combination of both. When implemented as a wirelesssystem, system 700 may include components and interfaces suitable forcommunicating over a wireless shared media, such as one or moreantennas, transmitters, receivers, transceivers, amplifiers, filters,control logic, and so forth. An example of wireless shared media mayinclude portions of a wireless spectrum, such as the RF spectrum and soforth. When implemented as a wired system, system 600 may includecomponents and interfaces suitable for communicating over wiredcommunications media, such as input/output (I/O) adapters, physicalconnectors to connect the I/O adapter with a corresponding wiredcommunications medium, a network interface card (NIC), disc controller,video controller, audio controller, and the like. Examples of wiredcommunications media may include a wire, cable, metal leads, printedcircuit board (PCB), backplane, switch fabric, semiconductor material,twisted-pair wire, co-axial cable, fiber optics, and so forth.

The thresholded pixel value matching and associated object processescomporting with exemplary embodiments described herein may beimplemented in various hardware architectures, cell designs, or “IPcores.”

As described above, system 600 may be embodied in varying physicalstyles or form factors. FIG. 7 further illustrates embodiments of amobile handset device 700 in which system 600 may be embodied. Inembodiments, for example, device 700 may be implemented as a mobilecomputing handset device having wireless capabilities. As shown in FIG.7, mobile handset device 700 may include a housing with a front 701 andback 702. Device 700 includes a display 704, an input/output (I/O)device 706, and an integrated antenna 708. Device 700 also may includenavigation features 712. Display 704 may include any suitable displayunit for displaying information appropriate for a mobile computingdevice. I/O device 706 may include any suitable I/O device for enteringinformation into a mobile computing device. Examples for I/O device 706may include an alphanumeric keyboard, a numeric keypad, a touch pad,input keys, buttons, switches, microphones, speakers, voice recognitiondevice and software, and so forth. Information also may be entered intodevice 700 by way of microphone (not shown), or may be digitized by avoice recognition device. Embodiments are not limited in this context.Integrated into at least the front 701, or back 702, are cameras 705,710 (e.g., each including a lens, an aperture, and an imaging sensor),each of which may be components of one or more CM through which imagedata is exposed and output to the point-to-point measurement device asdescribed elsewhere herein.

As exemplified above, embodiments described herein may be implementedusing hardware elements, software elements, or a combination of both.Examples of hardware elements or modules include: processors,microprocessors, circuitry, circuit elements (e.g., transistors,resistors, capacitors, inductors, and so forth), integrated circuits,application specific integrated circuits (ASIC), programmable logicdevices (PLD), digital signal processors (DSP), field programmable gatearray (FPGA), logic gates, registers, semiconductor device, chips,microchips, chip sets, and so forth. Examples of software elements ormodules include: applications, computer programs, application programs,system programs, machine programs, operating system software,middleware, firmware, routines, subroutines, functions, methods,procedures, software interfaces, application programming interfaces(API), instruction sets, computing code, computer code, code segments,computer code segments, data words, values, symbols, or any combinationthereof. Determining whether an embodiment is implemented using hardwareelements and/or software elements may vary in accordance with any numberof factors considered for the choice of design, such as, but not limitedto: desired computational rate, power levels, heat tolerances,processing cycle budget, input data rates, output data rates, memoryresources, data bus speeds and other design or performance constraints.

One or more aspects of at least one embodiment may be implemented byrepresentative instructions stored on a machine-readable storage medium.Such instructions may reside, completely or at least partially, within amain memory and/or within a processor during execution thereof by themachine, the main memory and the processor portions storing theinstructions then also constituting a machine-readable storage media.Programmable logic circuitry may have registers, state machines, etc.configured by the processor implementing the computer readable media.Such logic circuitry, as programmed, may then be understood to bephysically transformed into a system falling within the scope of theembodiments described herein. Instructions representing various logicwithin the processor, which when read by a machine may also cause themachine to fabricate logic adhering to the architectures describedherein and/or to perform the techniques described herein. Suchrepresentations, known as cell designs, or IP cores, may be stored on atangible, machine-readable medium and supplied to various customers ormanufacturing facilities to load into the fabrication machines thatactually make the logic or processor.

While certain features set forth herein have been described withreference to embodiments, this description is not intended to beconstrued in a limiting sense. Hence, various modifications of theimplementations described herein, as well as other implementations,which are apparent to persons skilled in the art to which the presentdisclosure pertains are deemed to be within the spirit and scope of thepresent disclosure.

The following paragraphs briefly describe some exemplary embodiments:

In one or more first embodiments, a computer implemented method forpoint-to-point measurement of image data includes receiving anindication of a pair measurement points within an input image frame,determining a 3D spatial coordinate for each of a plurality of pointsalong a measurement line defined by the pair of measurement points,filtering the 3D spatial coordinates associated with the measurementline to account for outliers, mapping the pair of measurement points toa pair of predicted points based on a function of the filtered 3Dspatial coordinates, determining a distance between the pair ofpredicted points, and storing the distance to a memory in associationwith the pair of measurement points.

In furtherance of the first embodiments, filtering the 3D spatialcoordinates further comprises fitting a linear structure to the 3Dspatial coordinates. Mapping the pair of measurement points to the pairof predicted points is based on a function of the linear structure.Determining the distance between the pair of predicted points furthercomprises computing the Euclidean distance between the pair of predictedpoints.

In furtherance of the first embodiments immediately above, mapping thepair of measurement points to the pair of predicted points furthercomprises fitting a plane to the 3D spatial coordinates, and projectingthe pair of measurement points onto the plane.

In furtherance of the first embodiments, fitting the linear structurecomprises performing a robust total least squares (TLS) fit of the 3Dspatial coordinates associated with the measurement line.

In furtherance of the first embodiments immediately above, performingthe robust TLS fitting further comprises: determining a robust estimateof the multivariate location and scatter of the 3D spatial coordinatesassociated with the measurement line, and determining a principalcomponent based on an Eigen decomposition of the scatter.

In furtherance of the first embodiments immediately above, determiningthe robust estimate of the multivariate location and scatter furthercomprises computing the minimum covariance determinant (MCD) estimatorto find h observations of the 3D spatial coordinates associated with themeasurement line associated with a covariance matrix having the lowestdeterminant. Mapping the pair of measurement points to the pair ofpredicted points further comprises determining if the h observations areall on a hyperplane, and computing the predicted points based on apredicted depth associated with the hyperplane in response to the hobservations all being on the hyperplane, or projecting the measurementpoints onto a line in response to the h observations not all being on ahyperplane.

In furtherance of the first embodiments, the method further comprisesconstructing a pair of line segments, each extending the measurementline beyond one of the measurement points, determining a depthassociated with each of the line segments, and determining the distancebetween the measurement points independent of the pair of refined pointsin response to both a first of line segment depths being less than afirst predicted depth associated with a first of the predictedmeasurement points, and a second of line segment depths being less thana second predicted depth associated with a second of the predictedmeasurement points.

In furtherance of the first embodiments, the method further comprisesreceiving image frame data including a depth value associated with each2D spatial position within the image frame, displaying the image frameon a display screen, and receiving the indication of the pairmeasurement points further comprises receiving a user selectionindicative of two or more spatial coordinates within the image frame.

In furtherance of the first embodiments, the method further comprisescollecting the image frame data with at least one of: a plurality ofcameras; a time of flight camera; or a structured light illuminator.

In one or more second embodiments, a computerized imaging device,comprises an input port to receive an indication of a pair measurementpoints within an input image frame, and one or more processors coupledto the input port. The processors are to determine a 3D spatialcoordinate for each of a plurality of points along a measurement linedefined by the pair of measurement points, filter the 3D spatialcoordinates associated with the measurement line to account foroutliers, map the pair of measurement points to a pair of predictedpoints based on a function of the filtered 3D spatial coordinates, anddetermine a distance between the pair of predicted points. The devicefurther comprises a memory to store the distance in association with theinput image frame.

In furtherance of the second embodiments, to filter the 3D spatialcoordinates, the one or more processors are to fit a linear structure inthe 3D spatial coordinates, to map the pair of measurement points, theone or more processors are to map the measurement points to the pair ofpredicted points based on the linear structure in the 3D spatialcoordinates, and to determine the distance between the pair of predictedpoints, the one or more processors are to compute the Euclidean distancebetween the pair of predicted points.

In furtherance of the second embodiments immediately above, to map thepair of measurement points, the one or more processors are to compute afunction of a plane based on the 3D spatial coordinates, and project thepair of measurement points onto the plane.

In furtherance of the second embodiments, to fit the linear structure tothe 3D spatial coordinates, the one or more processors are to perform arobust total least squares (TLS) fit of the 3D spatial coordinatesassociated with the measurement line.

In furtherance of the second embodiments immediately above, to performthe robust TLS fit, the one or more processors are to determine a robustestimate of the multivariate location and scatter of the 3D spatialcoordinates associated with the measurement line, and determine aprincipal component based on an Eigen decomposition of the scatter.

In furtherance of the second embodiments immediately above, to determinethe robust estimate of the multivariate location and scatter, the one ormore processors are to compute the minimum covariance determinant (MCD)estimator to find h observations of the 3D spatial coordinatesassociated with the measurement line associated with a covariance matrixhaving the lowest determinant. To map the pair of measurement points tothe pair of predicted points, the one or more processors are todetermine if the h observations are all on a hyperplane, and compute thepredicted points based on a predicted depth associated with thehyperplane in response to the h observations all being on thehyperplane, or project the measurement points onto a line in response tothe h observations not all being on the hyperplane.

In furtherance of the second embodiments, the one or more processors arefurther to construct a pair of line segments, each extending themeasurement line beyond one of the measurement points, determine a depthassociated with each of the line segments, and determine the distancebetween the measurement points independent of the pair of predictedpoints in response to both a first of line segment depths being lessthan a first predicted depth associated with a first of the predictedmeasurement points, and a second of line segment depths being less thana second predicted depth associated with a second of the predictedmeasurement points.

In furtherance of the second embodiments, the device further includes atleast one of: a plurality of cameras, a time or flight camera, or astructured light source to collect a depth value associated with each 2Dspatial position within the image frame. The device further includes adisplay screen to display the image frame, and the device furtherincludes a user interface to receiving a user selection indicative oftwo or more spatial coordinates within the image frame.

In one or more third embodiments, one or more computer-readable storagemedia has instructions stored thereon, which when executed by aprocessor, cause the processor to perform a method comprising receivingan indication of a pair measurement points within an input image frame,determining a 3D spatial coordinate for each of a plurality of pointsalong a measurement line defined by the pair of measurement points,filtering the 3D spatial coordinates to account for outliers, mappingthe pair of measurement points to a pair of predicted points based onthe filtered set of 3D spatial coordinates, determining a distancebetween the pair of predicted points, and storing the distance to amemory in association with the pair of measurement points.

In furtherance of the third embodiments, the media further comprisesinstructions stored thereon, which when executed by the processor,further cause the processor to perform the method further comprisingfiltering the 3D spatial coordinates further comprises fitting a linearstructure to the 3D spatial coordinates, mapping the pair of measurementpoints to the pair of predicted points based on the linear structure,and determining the distance between the pair of predicted pointsfurther comprises computing the Euclidean distance between the pair ofpredicted points. Fitting the linear structure to the 3D spatialcoordinates comprises performing a robust total least squares (TLS) fitof the 3D spatial coordinates associated with the measurement line. TheTLS fit further comprising determining a robust estimate of themultivariate location and scatter of the 3D spatial coordinatesassociated with the measurement line, and determining a principalcomponent based on an Eigen decomposition of the scatter.

In furtherance of the third embodiments immediately above, determiningthe robust estimate of the multivariate location and scatter furthercomprises computing the minimum covariance determinant (MCD) estimatorto find h observations out of the 3D spatial coordinates associated withthe measurement line associated with a covariance matrix having thelowest determinant, and mapping the pair of measurement points to thepair of predicted points further comprises determining if the hobservations are all on a hyperplane, and computing the predicted pointsbased on a predicted depth associated with the hyperplane in response tothe h observations all being on the hyperplane, or projecting themeasurement points onto a line in response to the h observations not allbeing on the hyperplane.

In one or more fourth embodiments, a computerized imaging devicecomprises a means to perform any one of the first embodiments.

In one or more fifth embodiment, a computer-readable storage media hasinstructions stored thereon, which when executed by a processor, causethe processor to perform any one of the first embodiments.

It will be recognized that the embodiments are not limited to theexemplary embodiments so described, but can be practiced withmodification and alteration without departing from the scope of theappended claims. For example, the above embodiments may include specificcombination of features. However, the above embodiments are not limitedin this regard and, in embodiments, the above embodiments may includeundertaking only a subset of such features, undertaking a differentorder of such features, undertaking a different combination of suchfeatures, and/or undertaking additional features than those featuresexplicitly listed. Scope should, therefore, be determined with referenceto the appended claims, along with the full scope of equivalents towhich such claims are entitled.

What is claimed is:
 1. A computer implemented method for point-to-pointmeasurement of image data, the method comprising: receiving anindication of a pair measurement points within an input image frame;determining a 3D spatial coordinate for each of a plurality of pointsalong a measurement line defined by the pair of measurement points;estimating a multivariate location and scatter of the 3D spatialcoordinates associated with the measurement line by computing a minimumcovariance determinant (MCD) estimator for the 3D spatial coordinates,the MCD identifying a covariance matrix having the lowest determinantfor a predetermined threshold number of the 3D spatial coordinates;determining if the predetermined threshold number of the 3D spatialcoordinates are all on a hyperplane; computing a pair of predictedpoints by: evaluating an equation of the hyperplane at the measurementpoints in response to the predetermined threshold number of the 3Dspatial coordinates all being on the hyperplane; or projecting themeasurement points onto a principal component determined from an Eigendecomposition of the scatter, in response to the predetermined thresholdnumber of the 3D spatial coordinates not all being on the hyperplane;determining a Euclidean distance between the pair of predicted points;and storing the distance to a memory in association with the pair ofmeasurement points.
 2. The method of claim 1, further comprising:constructing a pair of line segments, each extending the measurementline beyond one of the measurement points; determining a depthassociated with each of the line segments; and determining the distancebetween the measurement points independent of the pair of predictedpoints in response to both a first of line segment depths being lessthan a first predicted depth associated with a first of the predictedmeasurement points, and a second of line segment depths being less thana second predicted depth associated with a second of the predictedmeasurement points.
 3. The method of claim 1, further comprising:receiving image frame data including a depth value associated with each2D spatial position within the image frame; displaying the image frameon a display screen; and wherein receiving the indication of the pairmeasurement points further comprises receiving a user selectionindicative of two or more spatial coordinates within the image frame. 4.The method of claim 1, further comprising: collecting the image framedata with at least one of: a plurality of cameras; a time of flightcamera; or a structured light illuminator.
 5. A computerized imagingdevice, comprising: an input port to receive an indication of a pairmeasurement points within an input image frame; one or more processorscoupled to the input port, the processors to: determine a 3D spatialcoordinate for each of a plurality of points along a measurement linedefined by the pair of measurement points; estimate a multivariatelocation and scatter of the 3D spatial coordinates associated with themeasurement line from the minimum covariance determinant (MCD) estimatorcomputed for the 3D spatial coordinates, the MCD identifying acovariance matrix having the lowest determinant for a predeterminedthreshold number of the 3D spatial coordinates; determine if thepredetermined threshold number of the 3D spatial coordinates are all ona hyperplane; compute a pair of predicted points from: an evaluation ofan equation of the hyperplane at the measurement points in response tothe predetermined threshold number of the 3D spatial coordinates allbeing on the hyperplane; or a projection of the measurement points ontoa principal component determined from an Eigen decomposition of thescatter, in response to the predetermined threshold number of the 3Dspatial coordinates not all being on the hyperplane; and determine aEuclidean distance between the pair of predicted points; and a memory tostore the distance in association with the input image frame.
 6. Thedevice of claim 5, wherein the one or more processors are further to:construct a pair of line segments, each extending the measurement linebeyond one of the measurement points; determine a depth associated witheach of the line segments; and determine the distance between themeasurement points independent of the pair of predicted points inresponse to both a first of line segment depths being less than a firstpredicted depth associated with a first of the predicted measurementpoints, and a second of line segment depths being less than a secondpredicted depth associated with a second of the predicted measurementpoints.
 7. The device of claim 5, further comprising: at least one of: aplurality of cameras, a time or flight camera, or a structured lightsource to collect a depth value associated with each 2D spatial positionwithin the image frame; a display screen to display the image frame; anda user interface to receiving a user selection indicative of two or morespatial coordinates within the image frame.
 8. One or morenon-transitory computer-readable storage media, with instructions storedthereon, which when executed by a processor, cause the processor toperform a method comprising: receiving an indication of a pairmeasurement points within an input image frame; determining a 3D spatialcoordinate for each of a plurality of points along a measurement linedefined by the pair of measurement points; estimating a multivariatelocation and scatter of the 3D spatial coordinates associated with themeasurement line by computing a minimum covariance determinant (MCD)estimator for the 3D spatial coordinates, the MCD identifying acovariance matrix having the lowest determinant for a predeterminedthreshold number of the 3D spatial coordinates; determining if thepredetermined threshold number of the 3D spatial coordinates are all ona hyperplane; computing a pair of predicted points by: evaluating anequation of the hyperplane at the measurement points in response to thepredetermined threshold number of the 3D spatial coordinates all beingon the hyperplane; or projecting the measurement points onto a principalcomponent determined from an Eigen decomposition of the scatter, inresponse to the predetermined threshold number of the 3D spatialcoordinates not all being on the hyperplane; determining a Euclideandistance between the pair of predicted points; and storing the distanceto a memory in association with the pair of measurement points.
 9. Themedia of claim 8, further comprising instructions to cause theprocessors to perform a method further comprising: constructing a pairof line segments, each extending the measurement line beyond one of themeasurement points; determining a depth associated with each of the linesegments; and determining the distance between the measurement pointsindependent of the pair of predicted points in response to both a firstof line segment depths being less than a first predicted depthassociated with a first of the predicted measurement points, and asecond of line segment depths being less than a second predicted depthassociated with a second of the predicted measurement points.
 10. Themedia of claim 8, wherein receiving the indication of the pairmeasurement points further comprises receiving a user selectionindicative of two or more spatial coordinates within the image frame.11. The media of claim 8, further comprising instructions to cause theprocessors to perform a method further comprising collecting the imageframe data with at least one of: a plurality of cameras; a time offlight camera; or a structured light illuminator.